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The Emergence of Technological Tools Against Foreign 


Propaganda and Disinformation Online 


One of the most consequential events in modern Information Warfare occurred on 
February 3, 2015. 


Throughout 2014, the terrorist group Islamic State ISIS enjoyed seemingly continuous 
battlefield victories and manufactured a popular narrative of “remaining and 
expanding.”! They had taken the cities of Sinjar and Kobani, demonstrating their intent 
to develop a geographic caliphate, and along the way they tweeted constantly, earning a 
reputation for torturous executions, rape and enslavement - all of which triggered 
supportive audience reactions.* Researchers estimated 46,000 mostly uncontested 
ISIS-controlled accounts on Twitter, posting regularly, with about 1,000 supporters 
each. 3 


In December 2014, a Royal J ordanian Air Force F16 pilot named Muath Safi Yousef al- 
Kasasbeh crashed his jet near Raqqa and was captured by ISIS. ISIS initially used him 
in its Dabig magazine for propaganda, and then falsely offered to exchange him for ISIS 
prisoners. By J anuary, ISIS asked its Twitter followers for execution recommendations, 
using the hashtag #SuggestAWayTokillTheJ ordanianPilotPig to generate support. And 
by J anuary 3", ISIS propagandists recorded a theatrically orchestrated execution film in 
which Lieutenant al- Kasasbeh was burned alive in a cage, with cleanly uniformed ISIS 
militants positioned as a seemingly disciplined unit to witness the execution. 


ISIS released Lieutenant al- Kasasbeh’s execution video on February 3, 2015. Everyone 
who was involved in the Coalition effort to counter ISIS propaganda watched the video 
closely.5 It was remarkable for its orchestration and for its broad dissemination. The 
Middle East was captivated by the video, and although anti-ISIS viewers were enraged, & 
pro-ISIS tweets appeared to increase ten-fold in the month afterward, with ISIS’s 
capture of Tikrit in March bolstering their narrative.’ This was the peak of ISIS’s 
activity on social media and the pressure to counter them was acute. 
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Although the effort against ISIS’s online presence was underway, the al-Kasasbeh 
propaganda event hyper-focused policymakers on opportunities to undermine terrorist 
use of social media at scale.® It also reinforced the importance of Information Warfare, 
forcing the small Information Operations community to reconsider the importance of 
these tactics within the civil-military toolkit. And it encouraged the community to 
pursue solutions with social media industry partners: in the months after the al- 
Kasasbeh propaganda event, the community regularly and openly called upon Twitter, 
Facebook, and other platforms to suspend pro-ISIS accounts. The U.S. Government’s 
demands gained strength in the following months, culminating with a meeting of the 
Global Coalition Against ISIL Communications Working Group, including 
representatives of the major social media platforms, in October 2015.9 


The effort to suspend ISIS accounts on Twitter - which offers the most accessible data 
for analysis - paid off quantifiably, with “devastating reductions” in ISIS follower 
accounts from J une to October 2015. Average tweets per day per ISIS account during 
this period peaked at 14.5 per day in J une and dropped to 5.5 per day by October. 10 
Naysayers had derided these efforts as a game of “whack-a-mole” because ISIS Twitter 
accounts could be replicated rapidly and required repeated suspensions, !! but the effect 
was a net loss of ISIS social media presence. 


The success of ISIS account suspensions in those “early days” encouraged stronger 
efforts by social media companies acting on their own and according to their own terms 
of service. The U.S. Government took the position - as it does now - that it should not 
directly censor online communications, in support of free speech. Twitter reported 
suspending over 1,200,000 accounts between 2015 and 2018 for “violations related to 
the promotion of terrorism” and only .2 percent of those suspensions were based on U.S. 
Government reports. 13 


The ISIS propaganda effort persists today at a lower level, with decreased press 
attention.“ In the past few years, media attention seems to have shifted to Russian 
propaganda and disinformation activities on social media. The Russian Internet 
Research Agency (IRA) has conducted a complex campaign across multiple social media 
platforms reinforcing U.S. political divisions, peaking in 2017. For example, in 2017 the 
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IRA posted about 6,000 Instagram posts per month; about 4,000 Facebook posts per 
month; and over 59,000 Twitter posts per month. Over 30,000,000 users interacted 
with IRA content on Facebook and Instagram in 2015-2017.15 


Frustration with this abuse of social media by ISIS and state actors such as the IRA 
underlies recent calls for increased account suspension efforts by social media 
platforms.!6 Social media platforms commonly refer to these activities as “content 
moderation” - originally developed over a decade ago to combat copyright theft - and 
rely on Section 230 of U.S. Code 47 for “protection for private blocking and screening of 
offensive material.”!” In J uly 2018, Twitter released a suspended data set of 2,973,371 
tweets from 2,848 handles, run by the IRA from February 2012 to May 2018.18 These 
content moderation efforts have undoubtedly impacted Russian disinformation 
campaigns against U.S. audiences, but online disinformation campaigns by state and 


non-state actors continue. 


The success of content moderation against ISIS and the IRA is promising, but seems to 
have encouraged overreliance on this process as a singular solution to the problem of 
disinformation and propaganda efforts on the internet. For example, the May 2019 
Christchurch Call mandated a closer content moderation relationship between signatory 
governments and social media companies, with the goal of eliminating the kind of 
violent content that was posted on Facebook and 8chan before and during the March 15, 
2019, shooting attack at a mosque in Christchurch, New Zealand.!9 The Call seems to 
aspire to a “sophisticated algorithm” that ensures efficient suspensions, but Facebook’s 
recent explanation of their processes shows that content moderation is extraordinarily 
complex and doesn’t guarantee the kind of instantaneous results that would have been 
needed to prevent the dissemination of the Christchurch attacker’s propaganda.2° The 
video uploaded during the Christchurch attack could still be found online weeks 
afterwards. 
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The Promise of New Technologies to Address Audience 


Vulnerabilities to Propaganda and Disinformation 


Content moderation’s greatest use may be its contribution to addressing propaganda 
and disinformation in combination with other technologies that address the underlying 
psychological and sociological factors at play. One set of academic researchers tells us 
that news consumers tend to seek out information that aligns with their own views,?! 
while another group tells us that news consumers fall prey to disinformation because 
they are cognitively lazy.22 ‘These academic perspectives are refinements of enduring 
theories of human cognitive bias such as selective exposure, confirmation bias, and 
motivated reasoning. No matter which academic camp we choose, there are new 
technologies that can aid online information consumers by identifying the origins of the 
information they are seeing, offering context for the information, warning of artificiality 
of information, and arming them with better cognitive tools to judge information 


veracity. 
Content Validation 


The problems of disinformation and propaganda are often problems of attribution. In 
the case of videos and other imagery, propagandists sometimes misidentify the origins 
of an image in order to develop a false narrative about the image, and this 
misattribution threatens to confuse and misguide media consumers.23 One possible 
solution to the misattribution of content is blockchain-based information 

validation. Blockchain-based information validation systems digitally “fingerprint” a 
video or image when it is recorded and immutably store the fingerprint to the 
blockchain, so that anytime a user of that system accesses the video, they are assured of 
its origins. Insurers, for example, have begun to use these blockchain- based 
information validation systems to verify evidence of claims. Critics of these systems are 
quick to point out, though, that proving the origins of an image file does not prove the 
truth of the image itself, and that these systems are only useful for counter- 
disinformation if they are employed at scale. 
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De-Anonymized Social Media Accounts 


Most of the major social media platforms have experimented with account 

validation. Facebook and Twitter have used checkmark symbols to indicate whether an 
account holder has voluntarily documented their identity.24 This voluntary self- 
documentation by users has been mostly for reputation management by the famous, but 
could have advantages in countering disinformation originators if the process were 
popularized among all users. The process could be strengthened even more by 
encouraging social media users to feature their real names. Most of the Russian 
disinformation distributed online during the 2016 U.S. elections was distributed by 
fake-name accounts whose origins were indiscernible to average social media 

users.25 While account validation offers some clear opportunities against 
disinformation and propaganda, detractors argue that the anonymity of the internet is 
what makes it so successful, and that de-anonymization would undermine the fair use of 
the internet by otherwise suppressed voices. 


Automated Website Ratings 


A handful of self-described independent organizations have begun to market their 
services as neutral judges of disinformation originators online. These services are 
sometimes enhanced by artificial intelligence, which promises to instantaneously 
identify disinformation originators. These services are intended to help social media 
platforms avoid unintentionally amplifying false or problematic information and are 
intended to help advertisers avoid placing their ads on counterproductive 

websites. These services are not new, though: for over a decade, researchers have 
aggregated data on problematic ad originators and malware originators, and they have 
published their research openly to help content originators and advertisers avoid 
problematic associations.26 While these services may now carry the label of “black list,” 
any organization that has posted an ad online has indirectly used these services for 
years. 
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Crowdsourced Information Verification 


The internet often works to a propagandist’s advantage because it enables broad 
dissemination of information with little opportunity for direct refutation. As a result, 
media consumers may see only one side of an argument. Recent research indicates, 
however, that online groups can be effective judges of source reliability when they are 
primed to think critically, and can ably see through biased presentations.2” New 
crowdsourcing technology helps overcome the tyranny of one-sided presentations by 
providing a forum for critical "meta-conversations" - conversations occurring literally 
atop the original information, almost anywhere on the internet. This technology is 
currently popular with academics, who use it to discuss research papers posted online, 
for example. This technology requires much broader mainstream adoption, however, 
before it could counter propaganda and disinformation at scale. 


Online Advertisement and Content Attribution 


Under the assumption that malign influencers would take advantage of lax advertising 
standards to deliver disinformation in the run-up to European elections - as they have 
during previous elections2® - some social media providers recently began to require 
advertisers to clearly identify the origins of politics-related advertisements.292 By 
requiring advertisers to declare their allegiances with a "paid for by" statement, 
Facebook intended to address the problems of disinformation and propaganda on the 
supply side, before ads reach consumers - perhaps offering one of the most scalable 
solutions to the problem. Similarly, YouTube places overlay ads on videos it identifies 
as coming from state-funded broadcasters.°° Ad attribution, like other de- 
anonymization efforts, promises a new level of transparency, but for the time being the 
rules for these ads seem limited and circumventable. For example, an ad intended to 
influence voters during an election by focusing on voters' underlying psychological 
vulnerabilities - indirectly affecting their voting behavior - may evade new advertising 
transparency rules. 
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“Yellow Alerts” 


A number of analytic teams persistently research the online propaganda of groups like 
ISIS and the IRA, leveraging sophisticated social media monitoring systems, and they 
produce impressive reports of their findings. These reports typically are read only by a 
small audience, though, and the findings are lost on the majority of general internet 
users who might benefit from warnings of malign influence campaigns erupting 
online. Issuing succinct "yellow alerts" of these malign campaigns on social media 
platforms - where media consumers are most likely to encounter disinformation - 
could prepare consumers to encounter such campaigns critically. This concept often 
receives strong resistance, however, because the judgment of what is propaganda and 
what is truth can be subjective.3! Therefore, a warning system of this kind would only be 
politically feasible if it focused on objective facts of the deceptive techniques of a 
campaign. For example, a warning could describe ISIS's duplicitous use of false 
identities online, but a warning could not defensibly judge the truthfulness of an ISIS 
statement. However, social media platforms may hesitate to support a system that 
seems to imply a vulnerability of their own platforms. 


Tech-Enabled Media Literacy Training 


To judge by recent press, the public is mentally defenseless against online 
disinformation.°2 While this is an exaggeration, information consumers do sometimes 
fall prey to disinformation and propaganda because it is well hidden, seemingly 
indistinguishable from fact.33 Proponents of media literacy training promise to solve 
that problem by training and retraining information consumers in the art of factual 
discernment online. A handful of counter-disinformation games are online now, and a 
few non-profit organizations offer more intensive classroom and online training 
curricula. These games and classes take students through the process of applying 
rigorous sourcing standards to the information they are encountering, encouraging 
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stronger critical thinking. But perhaps the most limiting factor of these programs is 
their popularity - they remain “nice to have” features of education systems; 
furthermore, these effects of these programs will likely only be felt in the long term. 


The Complexity of Technology Implementation in the 
Context of the U.S. Constitution 


The U.S. government-industry relationship regarding online propaganda and 
disinformation has traditionally been informational. Both groups have relied upon 
formal information-sharing forums such as the Global Internet Forum to Counter 
Terrorism, the Global CT Forum, or Tech Against Terrorism to enable discussions of 
strategic intent. And when the U.S. Government is aware of an imminent threat of 
violence online, it may legally inform the social media platforms.** But the social media 
industry has historically appeared to keep governments at arm’s length to retain control 
of their own business processes and to reinforce their images as independent and free 
information-sharing platforms.®5 


The seven technologies above may offer mutually beneficial opportunities for 
collaboration between government and industry, with the goal of limiting the effects of 
online propaganda and disinformation. Together, these technologies avoid the trap of 
politically judging the content appearing online, they offer new market opportunities for 
social media platforms and related developers, and they offer the government an 
opportunity to demonstrate resolve by driving broad adoption. Perhaps most 
importantly, these technologies are market-driven, and are not punitive, encouraging 
continued growth of an important business sector. 


While the technological solutions provided here would intentionally support 
collaboration between government and industry, the international community seems to 
be heading in the other direction. Governments are enacting legislation that empowers 
them to punitively direct social media platforms to take down harmful content online. 
The definition of “harmful” is subjective and based on malleable government policy. 
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Singapore’s new Protection From Online Falsehoods and Manipulation bill, for example, 
enables the state to enforce removal of “false” content on apps such as WhatsApp or 
Telegram.26 Germany’s 2018 Network Enforcement Act holds social media companies 
responsible for content considered illegal under Germany’s existing hate speech laws, 
such as “incitement to hatred”.37 And the latest, the Christchurch Call, requires social 
media companies to voluntarily respond to reports of “terrorist and violent extremist” 


content online. 38 


These directive content moderation policies conflict with the U.S. public’s sense of free 
speech rights and even conjure nightmares of totalitarianism.°9 Yet commentators 
consistently complain of the threat of ceding online content review to a commercial 
industry. While the German Network Enforcement Act initially received strong 
constituent approval, the legislation now seems to be under litigative assault, perhaps 
demonstrating the limits of societal support.*° The reality of enforcement seems to have 
diverged radically from the original intent of the legislation, and that lesson is important 
to any U.S. technology implementation in this area: technology may be applied as a 
helpful tool for addressing skin-deep propaganda and disinformation symptoms, but 
not as a comprehensive solution to the deeper illness. 
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